Detecting medical status and cognitive impairment utilizing ambient data

ABSTRACT

Devices for determining the likelihood that a user of a primary monitoring device (“PMD”) has developed a medical condition, generally comprising sensors coupled to the PMD, wherein the PMD detects changes to sensor readings over time, and wherein the changes indicate a change in the likelihood that a user of the PMD has developed a medical condition. In some embodiments, motion sensors are operably coupled to the PMD, and the PMD monitors and saves data relating to characteristics of motion detected, which are used to determine whether there has been a change in a likelihood that a user is undergoing a medical event. In further embodiments, the PMD comprises cameras, and the PMD monitors and saves data relating to movement of a user&#39;s eyes, which is utilized to determine whether there has been a change to a likelihood that the user is currently undergoing a medical event.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and is a continuation of U.S. patentapplication Ser. No. 16/0117,100, filed Jun. 25, 2018, now issued asU.S. Pat. No. 10,342,463, which is a continuation of U.S. applicationSer. No. 14/658,074, filed Mar. 13, 2015, now issued as U.S. Pat. No.10,004,431, which claims priority pursuant to 35 U.S.C. § 119(e) to andthe benefit of U.S. Provisional Patent Application Nos. 61/952,759,61/952,781, 61/952,788, 61/952,792 and 61/952,799, all filed Mar. 13,2014. The text and contents of each of these applications are herebyincorporated into this application by reference as though fully setforth herein.

FIELD OF INVENTION

The subject disclosure generally relates to the field of medical andcognitive impairment. Specifically, embodiments of the present inventionrelate to systems, methods and devices for estimating the likelihood andlevel of medical or cognitive impairment of a person, and methods forcalibrating devices to measure the medical or cognitive impairment of aperson.

DISCUSSION OF THE BACKGROUND

For the purposes of this specification, the present invention willgenerally be described in relation to impairment caused by alcohol,drugs and medical conditions. However it should be understood that theinvention is not so limited, and may be applied and/or used to detectimpairment in a wide variety of other applications, including but notlimited to impairment due to toxic chemicals in the environment,chemical imbalances in body, changes to mental status, insufficientsleep, changes to blood sugar levels, divided attention, vision changes,hearing changes, use of a mobile device while walking or using avehicle, and/or other similar causes.

Humans have developed a society in which certain drugs (e.g., alcoholand prescription medications) are legal. There are restrictions on theuse of these drugs, and legal ramifications for exceeding the bounds ofthe law. Whether a human is within the legal limits of drug use or not,the consumer of the drug may experience effects of the drug. While thereare other physiological ramifications for drug use, the observableperformance effect that humans perceive in the drug consumer are aresult of the effect that the consumed drug has on the chemistry ofhuman brain. For instance, alcohol affects the brain by decreasing brainactivity using an inhibitory neurotransmitter. Concurrently alcoholcauses an increase in dopamine production resulting in a feeling ofpleasure. The inhibitory effects of alcohol affect the parts of thebrain responsible for movement, balance, sensory perception, reason andmemory.

For humans that have developed a dependence or addiction to drugs suchas alcohol, rehabilitation programs are available. Additionally, if aperson is brought up on charges for driving under the influence theoffender may find themselves on parole. The offender may also be orderedto wear an ankle monitor, or a SCRAM bracelet that is able to detectalcohol consumption. Breathalyzers and/or ignition interlocks areinstalled in the vehicles of some offenders. Currently available to manyconsumers are blood alcohol content calculating systems for mobiledevices to allow consumers to calculate whether or not the user hasreached the legal limit.

What does not exist in the art is a system or method for leveraging thesensors of a mobile device, such as a smartphone, to detect theperceived or actual intoxication of a human using the system or mobiledevice. In this digital age, mobile devices are taken with consumerseverywhere they go. Mobile device users are constantly interacting withtheir devices and even when they are not, the device is constantlysending, receiving and collecting data. Mobile devices have alreadyproven to be useful in athletic training and tracking, and these devicesfind more uses in the medical field every day.

In particular, drugs and alcohol are a significant public healthproblem. Many crimes, accidents, and injuries result from chemicalimpairment. Undesirable behavior frequently presents itself when humansare inebriated because many drugs, including alcohol, impair thefunction of the cerebral cortex of the frontal lobe, which isresponsible for the processing of information prior to acting. Someanimals, such as cats, are without this portion of the brain and, as aresult, immediate reaction to an action is observed in these species. Ina state of inebriation, a human may have slower than usual reactiontimes as their ability to process information decreases.

In addition, people may have reduced judgment when they are under theinfluence of alcohol or drugs, immediate irrational reactions once theyare severely inebriated and/or other impairments. Indeed, in some cases,the undesirable behavior includes the act of consuming the impairingchemicals. Moreover, impairment can occur without the involvement of anychemicals, such as in the case of changes to mental status, insufficientsleep, medical problems, changes to blood sugar levels, dividedattention, vision changes, hearing changes, use of a mobile device whilewalking or using a vehicle, and other causes. It should be appreciatedthat while this document references alcohol and chemical impairment, theinventions may be applied to other forms of impairment as well, such asthe ones in the preceding sentence, traumatic brain injury, excessivefatigue, stroke damage, and other similar impairments.

Many people have had the experience of observing a friend or anotherindividual become intoxicated and witnessed the change and the eventsthat may lead up to life altering mistakes. On the other hand, oncethese mistakes have been made and the offender is then mandated by thestate to reform, a monitoring system must be put in place to ensure thatthe judgment against the offender is upheld. In an effort to overcomethis monitoring system, some offenders remove tracking devices andemploy various schemes to obfuscate biological tests. In the midst ofthese efforts, it is often the case that, like most other people, theseoffenders keep a mobile device, such as a smartphone, with them nearlyall of the time. Indeed, in some cases the monitoring and/or abstinenceis voluntary, such as a person who is a recovering addict.

Due to the potential inability of a person to self-monitor or beaccurately monitored by a friend, and the ability of penal monitoringsystems to be foiled, at minimum it is desirable to employ a secondarysystem to monitor inebriation.

Applications also present in the medical field. After a procedure,patients are often prescribed pain killers containing codeine, Tylenol3, or opiates that encumber the patients' ability to operate machinerysuch as a motor vehicle. While not severely intoxicated, patients underthese circumstances could benefit from a personal monitoring system thataids the user in detecting a change in their normal performance.Prescription and other medications often come with a warning about therisk of impairment.

Finally, compliance with recommended medical testing is often difficultto obtain. Indeed, active participation in medical testing or monitoringis sometimes avoided because of the subconscious fear of a negativediagnosis. Therefore, it is desirable to perform medical diagnoseswithout the need for patients to take significant action or employspecialized equipment or tests.

There are significant public health and safety benefits to simplifyingthe detection of impairment, whether caused by alcohol, drugs, otherchemicals, mental status changes and/or medical issues. Existing methodsand devices for determining impairment rely on specialized diagnostictools, examination by specialists, or a combination of these methods.Consent to testing or examination is often difficult to obtain, andcompliance with recommendations that testing or examination be done isoften poor.

Consequently, there is a strong need for methods and devices that detectmedical and/or cognitive impairment without the need for specializeddevices, examination by specialists, consent of the person beingevaluated, and compliance with recommendations to be examined. To thisend, it should be noted that the above-described deficiencies are merelyintended to provide an overview of some of the problems of conventionalsystems, and are not intended to be exhaustive. Other problems with thecurrent state of the art and corresponding benefits of some of thevarious non-limiting embodiments may become further apparent upon reviewof the following description of the invention.

SUMMARY OF THE INVENTION

Embodiments of the present invention relate to methods and devices fordetermining the likelihood that a person is impaired and estimating thelevel of impairment of a person. The methods may be applied, at minimum,to determining the likelihood that a person is impaired due to alcohol,drugs, exposure to toxic chemicals, changes to mental status,insufficient sleep, medical problems, changes to blood sugar levels,divided attention, vision changes, hearing changes, use of a mobiledevice while walking or operating a vehicle. Other methods, devices andsystems for accomplishing similar objectives are disclosed in theco-pending application, Ser. No. 14/657,303, entitled “Systems, Devicesand Methods for Sensory Augmentation to Achieved Desired Behaviors orOutcomes,” filed concurrently by the inventors hereof, which is herebyincorporated by reference into this application as if fully set forthherein.

A system for detecting and approximating how intoxicated a human isbecoming can be useful in the consumer market, medical field andrehabilitation services. For example, due to the rhythmic nature of someaspects of human activity such as walking and talking, the actionsproduce a wave-like function that can be measured and recorded usingsensors on mobile devices. These activities are directly affected bydrug use, and as a result, how a human normally walks and talks, forinstance, may be compared to how the same person performs these sameactivities under the influence. Based on the data and the comparison ofthat data to “normal” or “baseline” data, the system may notify the useror system administrator of what may be signs of intoxication.

This ability to install a system that can tell users how drunk they mayappear may be useful for consumers, professionals and institutionsalike. Such system may be useful and easily configured for differentmarkets. The system may be useful to users who want to be responsiblewhile consuming legal drugs, and desire some assistance in monitoringthemselves as they use. The system may be useful to those consumers whohave identified a personal addiction problem. The system may also beuseful to rehabilitation facilities and law enforcement agencies. Thesystem may be ideal as a supplement to monitoring systems for parolees.The notification aspect of the system further enhances its utility asthe system may notify a parole officer of a violation, a sponsor of arelapse or a friend of another friend in need. Such notification may beaccomplished, among other mechanisms, via push notification.

In some aspects, a compelling feature of the present invention is thatthe mobile device has become so ubiquitous in society that for the vastmajority of people, their devices have become an extension of them. Thepresence of a passive health and intoxication monitoring system on theseever-present devices has the ability, at minimum, to inform users oftheir state of intoxication and, at maximum, save lives by disabling thecars of intoxicated drives and notifying friends.

Consumer devices have access to a remarkable amount of data measurableby the sensors natively present in them and the user data that thedevices process. By analyzing the data and comparing it to known orlearned data patterns, the likelihood that a user is impaired and/ormedically at risk may be evaluated. By gathering and analyzing ambientdata, a smartphone determines a “normal” data set for a user. Deviationfrom that data set is an indication of a possible problem. For example,if the motion sensing chip data indicates that a user's gait hassteadily become worse over the course of an hour while the user was,according to the GPS, proximate to a bar, the device may determine alikelihood that the user is inebriated. Changes to speech, sleeppatterns, time spent interacting with the device, the number of errorsthe user makes typing on the device, and other data are utilized todetermine the likelihood of a medical, mental or toxicologicalimpairment without the need for specialized equipment or tests.

In one embodiment, the invention relates to a method of estimating theimpairment of a person, the method comprising (a) gathering ambient datafrom one or more sensors operably coupled to a primary measuring device(PMD); (b) analyzing/identifying the data gathered; (c) querying atleast one data base to identify baseline data regarding the person; and(d) comparing the ambient data to the baseline data to determine alikelihood that the person is impaired. In some embodiments, the methodmay further comprise (e) gathering data from one or more externaldevices, and (f) comparing the external device data to the baseline datato further determine the likelihood that the person is impaired. The oneor more external devices may comprise, among other items, amotion-sensing watch, a head-mounted camera and/or a medical measurementdevice. The sensors may comprise a fingerprint sensor, an accelerometer,a three-axis gyroscope, an orientation sensor, an assisted GPS sensor,as well as others. In some embodiments, the method may further comprisea voice-to-text analysis of the person's speech, and/or identifyingpredictable causes to changes in ambient data to confirm or modify thelikelihood that the person is impaired.

The invention also relates to a method of calibrating a PMD, the methodcomprising (a) gathering calibration data about a person during one ormore calibration periods; and (b) analyzing the calibration data todetermine baseline data for the person, wherein the baseline data isconfigured to be compared against ambient data to determine a level ofimpairment of the person. In some embodiments, the method may alsocomprise (c) operably coupling a breath alcohol measuring device to thePMD, and (d) calibrating the PMD using the alcohol level measured by thebreath alcohol measuring device. In some instances, the method may alsocomprise (e) generating one or more profiles, and (f) comparing theambient data to the profile(s) to further determine the level ofimpairment of the person, wherein the profiles are generated byanalyzing data associated with one or more other persons with known orhighly likely states of impairment.

The invention further relates to devices to estimate the impairment of aperson, the device comprising (a) one or more sensors operably coupledto a first PMD, the sensors configured to capture ambient data about aperson, wherein the first PMD is configured to query at least one database to identify baseline data and compare the ambient data to thebaseline data to determine the likelihood that the person is impaired.In some embodiments, the first PMD may also share ambient data with asecond PMD and the second PMD may be configured to process the ambientdata shared by the first PMD. In yet other embodiments, the first PMDmay be configured to transmit notifications to one or more predeterminedcontacts.

Embodiments of the present invention advantageously provide methods,systems and devices for estimating the impairment of a person, withoutthe need for sophisticated medical equipment and/or specializeddiagnostic tools, examination by specialists, or some combinationthereof. Embodiments of the present invention also advantageouslyprovide methods and devices for determining the impairment of a personwithout the consent of the person being evaluated and/or compliance withrecommendations to be examined.

These and other advantages of the present invention will become readilyapparent from the detailed description below.

BRIEF DESCRIPTION OF THE DRAWINGS

Various non-limiting embodiments are further described with reference tothe accompanying drawings in which:

FIG. 1 schematically illustrates a method of determining the likelihoodof impairment of a person according to an embodiment of the presentinvention.

FIG. 2 schematically illustrates schematically illustrates a method fordetermining and modifying the likelihood of impairment based onpredictable changes to the ambient data of a person according to anembodiment of the present invention.

FIG. 3 schematically illustrates a method for determining the likelihoodof impairment of a person based on profiles generated by other impairedpersons according to an embodiment of the present invention.

FIG. 4 schematically illustrates a method of calibrating a primarymeasuring device, comprising an initial calibration phase and an ongoingcalibration phase according to an embodiment of the present invention.

FIG. 5 schematically illustrates active and passive calibration phasesaccording to an embodiment of the present invention.

FIG. 6 schematically illustrates a system, including various devices,for determining the likelihood that a person is impaired

DETAILED DESCRIPTION

Reference will now be made in detail to various embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. While the invention will be described in conjunction with thefollowing embodiments, it will be understood that the descriptions arenot intended to limit the invention to these embodiments. On thecontrary, the invention is intended to cover alternatives,modifications, and equivalents that may be included within the spiritand scope of the invention as defined by the appended claims.Furthermore, in the following detailed description, numerous specificdetails are set forth in order to provide a thorough understanding ofthe present invention. However, it will be readily apparent to oneskilled in the art that the present invention may be practiced withoutthese specific details. In other instances, well-known methods,procedures and components have not been described in detail so as not tounnecessarily obscure aspects of the present invention. Theseconventions are intended to make this document more easily understood bythose practicing or improving on the inventions, and it should beappreciated that the level of detail provided should not be interpretedas an indication as to whether such instances, methods, procedures orcomponents are known in the art, novel, or obvious.

For the sake of convenience and simplicity, the terms primary measuringdevice (PMD), smartphone, device, and mobile device may be usedinterchangeably herein, but are generally given their art-recognizedmeanings. Also, for convenience and simplicity, the terms user, subjectperson, consumer, and person may be used interchangeably, and whereverone such term is used, it also encompasses the other term.

As discussed in the background, there is a strong need for methods anddevices that detect medical and/or cognitive impairment without the needfor specialized devices, examination by specialists, consent of theperson being evaluated, and compliance with recommendations to beexamined. The present invention relates to methods, systems and devicesfor estimating the impairment of a person, without the need for suchsophisticated medical equipment, special diagnostic tools and/orexamination by specialists, or some combination thereof. Embodiments ofthe present invention also advantageously provide methods, systems anddevices for determining the impairment of a person without the consentof the person being evaluated.

Measurements of fine motor control, balance, speech slurring, speechpatterns and intonation, vital signs, nystagmus, pupil size, activity,lack of activity and response times can generate a fairly accurateapproximation of a user's level of intoxication and/or health.Similarly, location, word choice, behavior, amount and type of movement,frequency of bathroom use, changes to interactions with mobile devices,error rates in using mobile devices, types of responses tocommunications (e.g., screening phone calls), and proximity to certainother mobile devices or persons are all elements that may be associatedwith certain behaviors or conditions. Through calibration and/orartificial intelligence, a device may differentiate between a user'ssober and inebriated behavior and motor functions, as well as identifybehavior or changes to user behavior that may indicate medical problems.Indeed, in one aspect, the level of impairment and/or the level ofmedical risk may be approximated.

For purposes of this discussion, the term “primary measurement device”(“PMD”) refers to a device, in some cases a smart phone, that containsone or more sensors. Although the invention is often discussed here withreference to a smart phone, it should be understood that other devicesmay be included as well. For example, Google Glass®, which istechnically not a smart phone, has numerous sensors that would allow itto be utilized with regard to certain aspects of the present invention.Further, this document references a “mobile” device, but the mobility ofthe device is not necessarily required for at least some aspects of theinvention.

The primary measurement device may also be operably coupled, on atemporary or permanent basis, with additional devices, such as amotion-sensing watch, a head-mounted camera, or a specialized medicaldevice such as the Scandau Scout. The PMD may also be coupled with oneor more separate or integral processing units, and such coupling may beachieved directly, over a near field network (e.g., a Bluetooth), alocal area network, (e.g., a WiFi network), a cellular network, a widearea network, or otherwise.

There are a variety of sensors that are standard equipment on PMD's.Taking the Apple iPhone 5 as an example, the sensors (listed byfunction, whether or not their readings are generated by the samephysical chip) include: a fingerprint sensor; an accelerometer; athree-axis gyroscope; a compass; an assisted GPS; a front facing camera(and associated sensors); a rear facing camera (and associated sensors);a microphone; a digitizer (touch screen); an orientation sensor;cellular voice data; cellular data; WiFi data; Bluetooth data; acharging sensor; a battery measurement sensor; an ambient light sensor;a magnetometer; and a proximity sensor.

PMD's may additionally be equipped with a variety of other sensors,including, but not limited to: a wireless charging device (andassociated sensors); terrestrial/RDS/satellite radio sensors; pressuresensors; temperature sensors; humidity sensors; NFC communicationssensors; face and object detection (often within software supportingcamera); and/or barometric pressure sensors. PMD's may also have outputcapability, such as a light, that maybe used to enhance data gatheringby seasons. Further, PMD's may also have output capabilities, such as alight, which may be used to enhance data gathering by sensors.

PMD's may also receive, store, and send data, such as data generated asa result of interne usage, SMS usage, or other device usage. The searchand browsing history and/or text email data may be used to provide hintsto the PMD as to what impairment to test the user for, or to provide abasis to change the scaling of risks of impairment or illness. In somecases, the data may include data generated by specialized devices orexaminations, such as that generated by the Scandou Scout or a breathalcohol measurement device.

Chemical, mental state and medical impairment share manycharacteristics, and the description of these inventions in the contextof one form of impairment should be understood to apply to the others aswell. Because alcohol impairment is a fairly universal common point ofreference, alcohol impairment will be used frequently for purposes ofillustration. However, this should not be construed as limiting, as themethods and devices described herein apply to other chemical, mental andmedical impairment as well.

Exemplary Systems and Methods of Determining the Level of Impairment

In one implementation, a method of determining the level of impairmentof a person comprises (1) gathering ambient data from one or more of theavailable data sources operably coupled to a primary measuring device(PMD); (2) analyzing and identifying the ambient data; (3) querying atleast one data base to identify baseline data regarding the person; (4)comparing the ambient data to the baseline data, and (5) determining thelikelihood that the person is impaired. An embodiment of the method isshown schematically in FIG. 1.

In the embodiment of FIG. 1, smartphone (PMD) 105, having a plurality ofinternal sensors 108, is operably coupled to a first external sensor 106and a second external sensor 107, and also to a computing device 125.Although in the embodiment of FIG. 1, the PMD 105 is a smartphone, thePMD 105 may be another device, such as a tablet, a notepad, a laptop, apersonal digital assistant (PDA), or other devices, such as wearableubiquitous computing devices (e.g., Google Glass®), etc. The pluralityof internal sensors 108 may comprise a fingerprint sensor, anaccelerometer, a three-axis gyroscope, a compass, an assisted GPS, acharging sensor, a battery measurement sensor, an ambient light sensor,a magnetometer, an proximity sensor, an orientation sensor, one or morebiometric sensors, etc. Other data collection devices may also beintegral to the PMD 105. For example, the PMD may also comprise a frontfacing camera (with associated sensors), a rear facing camera (withassociated sensors), a microphone, a digitizer (e.g., a touch screen),etc. Further, the PMD 105 may have capabilities to capture other data,including, but not limited to cellular voice, cellular data (e.g., usagehistory), Wi-Fi data and/or Bluetooth data, and store and/or transmitdata.

The PMD 105 may optionally be equipped with one or more of a variety ofother sensors, including, but not limited to a wireless charging station(and associated sensors), terrestrial/RDS/satellite radio sensors,pressure sensors, temperature sensors, humidity sensors, NFCcommunications sensors, face and object detection devices (e.g., withinsoftware supporting camera), and/or barometric pressure sensors.

Each of first and second external sensors 106, 107 may be one of themany types of sensors listed above as sensors possibly internal to thePMD 105, except that external sensors 106 and 107 reside external to thePMD. In some instances, these external sensors already exist in thesubject environment (e.g., temperature, humidity sensors may exist intemperature controlled environments) and these external sensors may beleveraged for data collection where appropriate. Additionally, andalthough not shown in the embodiment of FIG. 1, external measurementdevices may also optionally be coupled to PMD 105. Such external devicesmay include, but are not limited to devices to measure, blood alcohollevel, blood sugar level, blood pressure, heart rate, activity level,calorie consumption, etc.

In the embodiment of FIG. 1, the PMD is operably coupled to a computingdevice 125, in which data may be analyzed, identified and/or stored.However, in other embodiments, the data may be analyzed, identifiedand/or stored on PMD 105 and/or PMD 105 may be operably coupled to oneor more servers or other computing/storage devices (e.g., a laptop,notepad, tablet, zip drive, thumb drive, CD ROM, DVD, etc.) from whichthe PMD may pull useful data.

The method 100 of FIG. 1 begins at step 110, where ambient data isgathered from internal and/or external sensors/devices. At step 120, thedata is analyzed and/or identified (e.g., by image, sound, odor,chemical and/or tactile recognition software). At step 130, one or moredatabases are queried to identify baseline data for the person for whomthe level of impairment is to be determined. At step 140, the ambientdata is analyzed and compared to the baseline data, and at step 150 thelikelihood that the person is impaired is determined.

In one aspect, device data may be utilized to generate a likelihood thatthe user is impaired, such as GPS data indicating that the user is in abar, voice-to-text analysis that indicates the user is discussingcurrent impairment or alcohol consumption, camera data showing that theuser is consuming alcohol, or other data. In one aspect, the user'slikely blood alcohol level may be estimated by analysis of the data(e.g., visual data) to determine the amount of alcohol consumed, theamount of food likely present in the user's stomach, etc. Other factorsrelevant to medical models for alcohol absorption and metabolism, suchas the user's percentage of body fat, weight and gender, as well as theamount of time since the user started drinking, may be identified aswell.

Similarly, where ambient data exists that would directly (or nearlydirectly) indicate the user's level of impairment, such data may be usedboth to determine the actual level of impairment and to calibrate howthe system utilizes other data to determine impairment for past orfuture periods. An example is that the user blows into akeychain-mounted breathalyzer and the results are observable by thedevice (e.g., by a camera mounted on the device).

In some embodiments, additional measurement devices may be utilized withor without the permission of the user of the PMD or the additionaldevices. The data generated by the additional devices may be gathered inwhatever manner in which the additional device makes the data available.In one aspect, the data may be obtained utilizing analog to digital datasensors (such as a CCD or CMOS camera) that, in many aspects, obtain thedata in the same way it is intended to be obtained by a user of theadditional device. For example, a blood pressure monitoring cuff mayhave no networking capability at all, but does present data via adisplay for perception by the human eye. By imaging the display, the PMDeffectively becomes a data recipient for the additional device. Itshould be further appreciated that the additional data may take the formof data generated entirely without computer involvement, such as streetsigns (for location data), a manual thermometer (for temperature data),or a clock (for time data).

In another aspect, multiple PMD's may share data, such as video streams.Such sharing may be of raw data, of data processed to indicate certaininformation, or a combination. In one example, two people may each bewearing a PMD with hardware similar to Google Glass®. The second PMDwould have a direct camera view of the user of the first PMD. The secondPMD may share a direct video stream.

In some cases, such as where bandwidth is limited, or where there is nobandwidth, data may be stored until a network connection is established.Where the permissions between the two PMD's indicate that raw data isnot to be shared, or otherwise, the second PMD may process the data(e.g., user of PMD 1 just took a drink of Coors Light® beer, thecontainer volume is 12 ounces, the container was 68% full previously,and the container is now 60% full). In some aspects, the first PMD maytell the second PMD what kind of data it needs. For example, the firstPMD may indicate that it needs to know whether the person is consumingdrinks purchased by another, so that the first PMD may more accuratelyrecord the amount of alcohol the person in consuming.

In another aspect, there may be a database which the second PMD mayconsult to identify the data. For example, if the person is at adrinking establishment with a group of friends, the second PMD mayaccess one or more databases to identify the individuals within thegroup. This aspect may be useful in a situation where a notification ofthe level of intoxication of a person is to be sent to at least one ofseveral friends. The second PMD may identify that only one friend in thenotification group is not participating in the current consumption ofalcohol, and may notify the non-participating friend to be on standbyfor notifications.

In another aspect, a payment may be made, or a credit useful forsomething of value may be created, in favor of a PMD that performs atask on behalf of a user of another PMD. Pooled or shared media, socialnetwork status updates, and other data sources may be utilized as well,for example, to identify useful information about the person or person'snetwork of friends, or to accept credits or payments for the tasksperformed by non-primary PMD's.

Whether calibrated against known periods of intoxication or not, the PMD105 may analyze the data available to determine “normal” or “standard”baseline data for a user. For example, the user's gait, the speed of theuser's pupillary constriction when transitioning from a dark to abrighter environment, the speed and/or auditory volume of a user'sbreathing, the rate at which the user blinks, the frequently of highacceleration events and/or other measurements are likely to fall withinone or more sets of regularly observed patterns. Such patterns may besegregated by time of day, by location (e.g., gait at the gym is likelyto differ from gait at home), by the task the user is doing (e.g., theuser's frequency and rate of eye movement is likely to differ whenplaying a fast-moving video game than when watching the evening news),etc. In one aspect, a set of standard measurements may be generated anda set of exceptions may be generated, the set of exceptions optionallygenerated at least in part by the segregated patterns.

In one aspect, predictable causes of changes to behavior may beidentified and accounted for in data analysis. For example, apremenopausal woman may experience a monthly cycle that includeshormonal changes, changes to facial and soft tissue symmetry, changes tothe number of visits to the restroom, and the potential presence ofblood in the toilet (which, for a male user, may be an indicator of amedical problem that a camera-based data source for a PMD might detect).Another cyclic cause of change is the work week when compared to theweekend.

One aspect of the present invention includes a method for accounting forpredictable changes in behavior when analyzing the likelihood that aperson is impaired. Such aspect is schematically illustrated in method200 of FIG. 2. Method 200 begins at step 210, where ambient data about aperson is gathered. The ambient data may be gathered by any number ofdifferent sensors (both internal to a PMD and/or external), and/ordevices which provide additional ambient/medical data as is describedabove.

At step 220, the captured data is processed and images, sounds, odors,etc. are identified (e.g., by image, sound, odor, chemical and/ortactile recognition software). At step 230, one or more databases arequeried to identify baseline data about the user. At step 240 theambient data is analyzed and compared to the baseline data. At step 250,based upon the comparison of step 240, the likelihood of impairment isdetermined. The method then proceeds to step 260, wherein one or moredatabases are queried to identify predictable causes and/or changes inambient data. At step 270, a determination is made as to whether theambient data correlates to a predictable change. If “no” then the methodends at step 275, and the initially determined likelihood of impairmentis the final determined likelihood of impairment. If, instead, thedetermination made at step 270 is that “yes,” the ambient datacorrelates to a predictable change, then at step 280, the likelihood ofimpairment is modified based on the predictable change in ambient data.

In one aspect, data analysis (e.g., at step 140 in method 100 of FIG. 1,or at step 240 in method 200 in FIG. 2) may be accomplished usingalgorithms similar to those used by email filtering systems, such as theGmail™ spam filtering system. Using such a system as an analogy,providing data generated during periods of known intoxication may besuperficially similar to seeding the system or correcting the system ina manner superficially analogous to manually identifying spam andnon-spam emails. However, the system may function without such seeding,and, over time, may reach an accuracy level nearly identical to thatpossible with seeding.

In one aspect, Bayesian filtering or other data analysis may be donewhereby data associated with one or more people with known (or likely,or highly likely) conditions or states of intoxication are utilized togenerate profiles that can be used to measure the likelihood that adifferent user is experiencing similar conditions. In another aspect,the changes between baseline readings for one or more people with known(or likely, or highly likely) conditions or states of intoxication andtheir readings when experiencing the effects of the conditions or statesof intoxication may be utilized to generate a profile for changes inreadings from baseline (or otherwise) that indicate a probability that auser is experiencing a similar condition or state of intoxication.

Referring now to FIG. 3, therein is shown a schematic representation ofa method 300 in which one or more profiles generated from other people303 are utilized to determine the likelihood of impairment of a user301. Method 300 starts at step 305, wherein data is gathered from otherpeople 303 with known or highly likely impairments. Such data may begathered using one or more of the internal/external sensors and/ordevices described above for methods 100 and 200. At step 315, the datais analyzed and profiles are generated based on such data. Such profilesmay correlate the data gathered from the other people 303 with theintoxication level of such other people 303.

At step 310, ambient data about the user 301 is gathered. As with step305, the ambient data may be gathered using one or more of theinternal/external sensors and/or devices described above for methods 100and 200. At step 320 the ambient data is analyzed and/or identified(e.g., by devices and methods described above for FIGS. 1 and 2). Itshould be noted that steps 310 and 320 may be performed after orsimultaneously with steps 305 and 315.

Typically, steps 305 and 315 will be performed before ambient data aboutthe user 301 is gathered (before step 310), and the resultant profileswill be stored in one or more databases for future access. Any number ofprofiles may be generated, and each such profile may be generated fromone person, or by averaging the data from a plurality of persons.

At step 330, ambient data about the user 301 is analyzed and compared tothe profiles generated from the other users 303. Then, at step 340, thelikelihood that the user is impaired is determined from the comparisonto the profiles. In some instance, not only is the likelihood ofimpairment determined, but also the level of impairment. For example, inreference to impairment due to alcohol consumption, the system mayindicate that the person is slightly impaired, but likely under thelegal BAC limit for operating a motor vehicle, or that the person isvery impaired at a level likely twice the legal BAC limit for operationof a motor vehicle.

The set of other users 303 utilized to generate the profile(s) may, insome aspects, be selected based on similarities to the user 301 beingevaluated. For example, evaluation of a male user in his mid-20s whoweighs 200 pounds and is 5′10″ tall may result in creation of one ormore profiles based on data for other users who share the same orsimilar characteristics.

In certain aspects, confidence intervals may be utilized. For example,it may be desirable to require a 2 sigma deviation over one episode or a1 sigma deviation over ten different episodes before indicating that theuser 301 should be tested, for example, for inner ear balance issues. Inanother example, the duration of male urination may be required to be a1 sigma deviation 5 times or 2 sigma deviations 2 times before takingaction based on risk of prostate enlargement, particularly when theurination sound is weaker or quieter or starts and stops or takes longerthan expected to stop.

In one aspect, biometric and similar data may be utilized to verify theidentity and/or identify the activities of the user 301. For example,the inventions may monitor purchases via near field communication,snooping via Wi-Fi unencrypted packets in promiscuous mode, or othersimilar mechanisms/methods. In one aspect, monitoring may focus onpurchases of health-related and/or unhealthy items (e.g., alcohol), andmay be utilized as additional data to measure and/or determineimpairment to health. Utilizing the alcohol purchase example, such datamay be utilized to improve the prediction and measurement of howintoxicated a user is likely to be. When, for example, purchases aremade via a purchasing/spending app or function of the PMD, banking datais available on the PMD, and/or the PMD receives an authenticationrequest (e.g., by SMS) to confirm a purchase is legitimate, such datamay also be utilized in determining the likelihood that the user isimpaired. In addition, data from financial transactions, insuranceclaims, GPS, and other sources may be correlated with pictures, socialmedia, and other data sources to determine or approximate the mentalstate the user was likely in at the time certain measurements were madeand/or purchases detected.

Exemplary Methods of Calibrating a PMD to Determine Impairment

The present invention also relates to a method of calibrating a PMD, themethod generally comprising (a) gathering calibration data about aperson during one or more calibration periods; and (b) analyzing thecalibration data to determine baseline data for the person, wherein thebaseline data is configured to be compared against ambient data todetermine a level of impairment of the person.

Normal (baseline) behavior and movement of the user may, in someaspects, be determined by the PMD through a combination of calibrationand learning. Calibration may be used to facilitate the learning of thePMD. Calibration and learning may be, in some cases, not significantlydifferent. In regards to some embodiments, calibration may be an initialperiod of time or amount of data necessary for the PMD to model anappropriate distribution for each behavior being monitored by the PMD.

In one aspect, before beginning the calibration period, if the user ofthe mobile device has been mandated by a court to utilize the device,the device may pull data, such as age, gender, height, weight, and racefrom penal system servers. If the use of the system is voluntary, theuser may enter or edit this data from the settings interface of thesystem. This data may improve the accuracy of the system as it willallow the system to use the data gathered from external sources asparameters to create a context for the calibration and learning of thePMD. It should be noted that these examples are not intended to belimiting, and there will be numerous additional situations in which datamay be mandated to be entered, data may be voluntarily entered, data maybe drawn from other sources (such as social media), or otherwise.

Referring to FIG. 4, therein is shown an exemplary method 400 ofcalibrating a

PMD. In the embodiment of FIG. 4, the method comprises an initialcalibration phase 401 and an ongoing calibration phase 499. The initialcalibration phase 401, begins at step 410, wherein calibration data isgathered from a smartphone (PMD) 405 having internal sensors 408, abreathalyzer 406, a server 407, and a user 409. The internal sensors 408may comprise any number of sensors as described in methods 200 and 300of FIGS. 2 and 3, respectively. Although the method of 400 of FIG. 4 isshown to gather data from internal sensors within the PMD 405, in otherembodiments, calibration data may also be gathered from any number ofexternal sensors as described in method 100 of FIG. 1 above. Likewise,and as described above in relation to FIG. 1, although the PMD of FIG. 4is shown as a smartphone, the PMD may be any device capable ofgathering, storing, analyzing and/or transmitting data.

At step 410 of the initial calibration phase 401, data is gathered froma breathalyzer 406. The breathalyzer 406, measures the actual level ofintoxication of the user 409. By correlating the measurements from theinternal sensors 408, as well as other data gathered by the PMD 405,with the blood alcohol level as measured by the breathalyzer 406, thedata generated by the sensors on the PMD 405 may be identified as beinggenerated while the user 409 had a set level of impairment. In someembodiments, a device that measures blood alcohol level may be part ofor integral to the PMD 405.

In addition, step 410 may comprise gathering data from one or moreservers (e.g., penal system servers, workplace servers, serversassociated with social media sites, cloud based servers, etc.) toobtains data regarding the user 409. Some servers may provide such datawithout requiring permission of the user 409, whereas otherservers/sites may require the user 409 to input username and/or passwordinformation in order to obtain access to the data. In some embodiments,medical data may be obtained regarding the user 409 with the appropriatepermissions.

As part of the initial calibration process 401, the user 409 may berequired to or may voluntarily input certain data. Such data maycomprise body weight, anatomical gender, age, height, body mass index(BMI), race and/or other data. The anatomical gender of the user 409 ispreferred over the gender that the user 409 identifies with, for thesake of anatomical and physiological accuracy. Blood alcohol content(BAC) tables are gender specific and as a result, the accuracy of thesystem in determining how easily the user 409 may become intoxicated ispartially dependent on the correct gender input. The inputted data maybe compared to a BAC table internal to the PMD 405, or a BAC tablestored on a device or database remote from the PMD 405.

Gender is a useful parameter as there significant disparity between menand women's susceptibility to intoxication. For example, medical modelsfor alcohol absorption indicate that premenopausal women get intoxicatedfaster than their male counterparts after drinking the same amount ofalcohol. Additionally, the general recommended servings of alcohol arebased on a 155-pound male having consumed three standard- sizedbeverages.

In addition to gender, when accounting for genetic or health baseddispositions, such as the onset of menses, age is also an importantparameter. For instance postmenopausal women metabolize alcohol at amuch slower rate than younger women. However, in general there is anegative correlation between alcohol metabolism and increase in ageacross both sexes. This can be attributed to the fact that, in general,both aging of the body and the brain increase the propensity of the user409 to experience the effects of chemical intoxication more rapidly.

Ethnicity is also a parameter that serves as an indicator of aconsumer's predisposition to become intoxicated. Especially with regardsto alcohol consumption, some ethnic groups (e.g., Asians) arepredisposed to a deficiency in a key enzyme necessary for alcoholmetabolism. In such ethnic groups, the lack of sufficient quantities ofthe enzyme acetaldehyde dehydrogenase, leads individuals in these groupsto experience symptoms that they may find unpleasant after consumingsmall amounts of alcohol, whereas individuals outside these groups donot experience similar unpleasant symptoms until they become heavilyintoxicated.

The size of the consumer, more specifically the body mass index (BMI) ofthe individual is a most critical parameter for which to account. Verysimply, the greater the body mass, the greater the volume of bloodnecessary to nourish that body, and as a result, a larger individual cantolerate the same amount of alcohol far better than their less massivecounterparts without feeling the effects. To calculate the BMI of theuser 409 the system may use the height and weight data pulled from aserver (e.g., server 407 of FIG. 4), derived from ambient data orsensors, or otherwise. This parameter is important because despite thefact that it is not completely accurate, it is the most consistentdeterminer across both genders. In other words, whether the user 409 isa big woman or a big man, they may feel the effects of intoxicationslower than a smaller woman or a smaller man, in general. This is truebecause with the addition of every two pounds of weight, the bloodvolume of an individual increases a little over one percent. Percentageof body fat is also a significant factor that may be utilized in asimilar manner. In one aspect, height, weight, and/or body fatpercentage may be approximated by analysis of data from ambient sensors,or other data. For example, the appearance of the user 409 in a mirroror in a social media networking photograph may be used, optionally inconjunction with other objects that provide a scale, to determineheight, weight and/or body fat percentage.

By obtaining such data, the device may be better able to determineadditional data that is required or desirable or may be better able todetermine the susceptibility of the user 409 to intoxication based oncertain amounts of chemicals. In essence, the data may be utilized toprovide a context by which the system may identify risks, identify datasets for comparison, and to otherwise analyze user information.

Referring again to FIG. 4, at step 420 calibration data is analyzed forpatterns, similarities, correlations, other indicators, etc., as well asdata that appears contrary to other data gathered that may suggest anerror or anomaly. At step 430, models are queried and the data gatheredis compared to charts and/or tables of impairment. At step 440, baselineimpairment data is determined for the user 409. Step 440 completes theinitial calibration stage. However, calibration and learning continue inthe ongoing learning/calibration phase 499. At step 460, ambient datacontinues to be gathered from PMD 405 and its internal sensors 408. Atstep 450, the ambient data is analyzed and/or compared to baseline data,and baseline data may be modified based on the additional ambient datagathered and analyzed in the ongoing learning/calibration phase 499.

Although in the embodiment of FIG. 4, the ongoing learning/calibrationphase 499 is only shown to modify baseline data based on additionalambient data gathered from the PMD 405, the ongoing learning/calibrationphase 499 may also comprise comparing and adjusting baseline data basedon additional data input by the user 409, additional data later acquiredfrom one or more servers (e.g., server 407 of FIG. 4), and/or additionaldata later gathered from one or more external measurement devices (e.g.,additional breathalyzer data, medical data from a blood glucose, BMI,blood pressure and/or heart rate monitor device, etc.)

In one aspect, during the device calibration phase or some other period,the user 409 may be required to walk (or measure while walking) a knowndistance and/or time interval with PMD 405 placed in a variety of knownpositions on the user's person. Alternatively, the PMD 405 may determineits position with some level of precision by utilizing measurements suchas camera, noise (i.e. proximity to voice, breathing, or feet walking),triangulated signals, or other data. In this way the PMD 405 may be ableto determine the normal gait of the user 409. For example, the PMD 405may determine the normal gait of a user 409 while walking with the PMD405 in his right pocket. In one aspect, the measurement may be based onwalking at least approximately 350 yards on at least seven days.

The purpose of this is multifaceted. In some aspects, it is beneficialfor the PMD 405 to be able to detect where the PMD (smartphone) 405 ison the user's person when a certain data input is observed for theuser's gait. This may enable the PMD 405 to determine what data shouldbe expected when the device is in a specific storage place on the user409. The data inputs from the different clothing pocket or other storagepoints may be jointly utilized to comprise the user's gait profile. Fora female, the smartphone may be placed in female specific storagelocations such as the woman's purse during calibration.

After calibration, the PMD 405 may continue to record the user's gait.Through this recording, the PMD 405 may continue to measure and presentan ever more accurate graphical representation of the user's gait. Thismeasurement process may allow the PMD 405 to know the gait of the ownerof the mobile device and differentiate the gait of its owner from otherpeople. Furthermore, this continued measuring may allow the PMD 405 toadjust to occurrences affecting gait such as crippling injury.Ultimately, the PMD 405 may allow the device to know how it shouldexpect to move if it is being carried by the user 409, and the certainstorage point that the PMD 405 is being carried on by the user 409.

If the present gait reading of the user 409 consistently falls outsideof the expected cycle function over a short interval of time, then thePMD 405 may return that the user's gait is abnormal and the user 409 maybe losing his or her fine motor coordination and balance. The reasonthat the abnormal gait must be consistently sustained for at least ashort interval of time is that the system must be able to overlookaccidental stumbles and blunders. When referring to the abnormal gait asbeing consistently sustained for a short interval of time, it is not tosuggest the abnormal gait pattern will return a wave-like function likethe normal gait. Rather, it is meant that the gait consistently fallsoutside what has been determined to be the normal gait of the user 409.In one aspect, different baselines or calibration or executed gait maybe utilized for different shoes or clothing.

It should be appreciated that while the discussion herein is with regardto measurement, calibration, and/or ongoing refinement of certainmetrics (such as gait), the discussion is intended to be illustrative ofother metrics as well. For example, the steadiness with which the PMD405 is held, the appearance of the user 409 in video, or other factorswould, in some aspects, be subject to a procedure similar to thatdescribed for gait.

Another phase of the initial and ongoing device data analysis refinementand calibration is facial recognition of a user. This calibration may beactive, passive, or a combination. The system 500 of FIG. 5schematically illustrates such active/passive calibration 550. In theembodiment if FIG. 5, the passive calibration phase 530 comprisesgathering data/user photos from a server 531, cloud-based storagesystems and social media 532, as well as gathering environmental data533. The active calibration phase 520 comprises gathering data fromdevices such as breathalyzer 521, data/photograph(s) from a userinterface 522 and photograph(s) and/or video from a camera 523.

In one embodiment, where use of the system 500 is mandated by penalcode, initial data may be obtained when system 500 pulls one or morephotos on file from a server 531 (e.g., a penal system server). In othercases, initial data/photo(s) may be obtained using a camera on the PMD510, a camera 523 (e.g. on another PMD), photograph(s) from social media532, photograph(s) stored on the PMD 510, or other sources. Theseinitial photo(s) may be utilized to provide the PMD 510 with an initialframe of reference from which to identify the user in subsequentlyanalyzed data.

Active calibration 520 may be available to the user from a userinterface 522 of the system. Such calibration may be desirable in aninstance in which, for some reason, the system 500 is having difficultyrecognizing the user, such as when analyzing low resolution media.Additionally if the user is using the system 500 voluntarily and thereis no initial photo for the system 500 to use as a reference, the system500 may prompt the user to either upload one from their gallery or takea new photo. At which point the system 500 may behave as it does duringactive calibration 520. The active component of the calibration 520would involve the user taking photos of themselves from different angleswith the camera 523 as they are prompted to do so by the system 500.

Additionally the system 500 may pull media, such as photos of the userfrom other locations such as, but not limited to, public media streams,social media, and other cloud based media services 532. In addition,photographs and other measurements may be obtained from various sources,some of which do not require permission of the user and/or permission ofthe photographic source. In addition, in some embodiments, the system500 may prompt the user to add data to photos as they are taken, such asidentifying certain features of the user (e.g., the user's pupils), toidentify the user's body parts (e.g., the user's hands), or other data.

In one aspect the system 500 may prompt the user for data input to helpthe system 500 make a decision such as identifying the user in a photo.For example, if photos or videos were captured at a family reunion, andthe PMD 510 was used to take a photo or video containing the user and asibling (or other person) with similar facial morphology, the system 500could prompt the user, asking the user which person in the media isthem. In another instance the system 500 may have difficulty locatingkey morphology that the facial, eye, pupil or other recognitionalgorithms may rely on and as a result the system 500 may prompt theuser to touch the tip of their nose or select their eyes for instance.

The passive phase of the calibration 530, which would occur by default,would comprise analyzing photos and videos in the user's photo galleryon their mobile device (e.g., their smartphone/PMD 510). The passivecomponent 530 of this calibration phase, like the active component 520,may allow the system 500 to learn what the user looks like. There aretwo reasons for the need for facial recognition capabilities in thesystem. The first reason is that the system 500 may need to tell theuser apart from other people in a photo. Secondly, once the system 500has determined who the user is in the photograph, the device may analyzefacial features to determine health or impairment data, and in order todo so, it must be able to locate the facial features.

For example, the device 510 may find the user's eyes so it can measurethe pupil dilation of the user. In order to accurately return meaningfuldata, for example, about the significance of the user's pupil dilation,the device 510 must be able to measure data source (for example, thechange in pupil size). As a result there may, in some cases, be aminimum desired mega pixel specification for the camera of the device510. In one aspect, the device 510 may aggregate a plurality of imagesin order to construct a high enough resolution composite image foranalysis.

Additionally, the device 510 may determine whether the pupil size of theuser is appropriate for the lighting of the environment that the user isin by gathering ambient data 533 in the passive calibration phase 530.Similarly, the device 510 may measure whether the user is listening tomusic or has the speaker for phone calls turned to a higher or lowervolume than is normal for the situation.

The system 500 may improve recognition of a user by pulling photos fromthe user's gallery on their device 510 (or from other sources) andtransforming the photo to different angles. In some cases, such as lightfield photography, such transformation may be accomplished utilizing asingle photograph. Additionally the system 500 may pull videos from thedevice 510 or other sources and analyze each frame of the video insearch of the user. When the user is found in a frame, the frame may betransformed like the standard photo. Once the face of the user has beendetected and tagged, the system 500 may locate the user's eyes using atemplate based method. From that point the center of the user's radiusmay be found.

In one aspect, how bright the user is keeping the display of the PMD 510may alert the system 500 of abnormal pupil function. Especially if theuser overrides the screen brightness determined by the operating systemof the PMD 510 by way of the ambient light sensor, the system 500 maycompare the manually set screen brightness to the suggested brightnessthat the operating system may default to. As a result the system 500 mayprompt the user with a pupil dilation test or a general eye movementtest.

One mechanism by which the device 510 may obtain additional data is bymeasuring factors that are apparent in stored or live streamed video,audio, or images. For example, a user's gait may be apparent in a videotaken and posted on Facebook™ or another social networking site. Byanalyzing the comments and other data related to the video, the system500 may be able to infer the state of health and/or inebriation at thetime the video was taken. The use of such external data may also behelpful in further calibration of the system 500 and/or in validatingthe inferences drawn from the motion data against actual video data. Insome instances, the time at which the image or video was captured may beavailable (for example via metadata, via a watch in the image, orotherwise), and actual data captured by the PMD 510 may be compared tothe image or video data in order to improve calibration of how the PMD510 interprets data it measures. The location of the PMD 510 on the usermay also be determined utilizing such external data sources.

Through the analysis of a number of photographs of the user in differentenvironments, the system 500 may be able to determine that in a certainknown lighting range, the user's pupil size should fall within a certainrange of diameters. If the pupil size falls outside of the range for aknown environment lighting range, the system 500 may return a messagestating that the pupil size is abnormal and something may be wrong withthe user.

An additional phase of device calibration may involve the devicelistening to the user as they speak, to learn how the user speaksnormally. As a result the system 500 may be able to detect changes inthe user's speech rate and volume. Once the system 500 has learned theway that the user speaks to a point of high accuracy, the system 500 maybe able to determine when the user is speaking at a slower rate or whenthe user is speaking consistently louder than usual. Speech patterns maybe analyzed with different people in different settings with differentexpectations. For example, a user may talk quietly and slowly with achild at night, but loudly and quickly while working at a restaurant.

The accuracy with which the system 500 can interpret what the user issaying may allow the system 500 to estimate when the user is slurringtheir speech, as the system 500 may not be able to understand the user.The system 500 may need to establish a voice profile for the user inwhich to store the properties of the user's speech. This may allow thesystem 500 to better ignore other unknown voices and ambient sounds.When the sounds of other voices and ambient sounds is too great and thesystem speech recognition accuracy has dropped critically low, therecognition feature may return an internal log stating that theenvironment is too loud, and suspend the speech recognition featureuntil the recognition ability accuracy of the system 500 reaches athreshold, such as one of 50% or more.

Features of the system that may not require calibration or may requireless calibration are the sobriety tests. When the system 500 detectsabnormal behavior, the system 500 may begin to time the periods betweenabnormal behavior readings. As the frequency of abnormal readingsincreases, the system 500 may increase the frequency with which it teststhe capability of the user. The system 500 may test the capability ofthe user by prompting the user to take tests such as a copying a line ofknown text with the keyboard of the mobile device 510. This may activelytest the user's fine motor function. The system 500 may also test theuser by requesting that the user speaks a line of known text into themicrophone of the mobile device 510. This may actively test the user'sability to speak. The system 500 may also test the user by flashing astring of four (or some other number of) characters on the screen of themobile device 510. This exercise may test the user's memory.

Further, the system 500 may prompt the user to follow a brightly coloredobject (such as a ball) on the screen of the mobile device 510, andusing the front facing camera and pupil detection algorithms, the system500 may be able to detect the speed of the user's pupil movement. Thismay detect the speed of eye movement which is an important determiner ofintoxication. In some aspects, the system 500 may be able to determinethe user's pulse using the mobile device's cameras and/or an externalheart rate monitor. Elevated and diminished pulses are good indicationsof intoxication when compared to the user's normal resting pulse. Itshould be appreciated that additional tests may be utilized and thedescription of tests herein is not limiting and is subject to change dueto the availability of updated or new sensors that may be leveraged togather more data or more accurate data.

In one aspect, the device 510 may recreate or emulate tests utilized bylaw enforcement to measure sobriety. For example, the device 510 may askthe user to walk in a straight line and utilize the ambient data sensorsto determine the user's success. The device 510 may identify an existingstraight line (such as the marking on a parking lot stall) or may, inthe case of devices that can project images such as Google Glass®,create a virtual line. Similarly, the device 510 may ask a user to saythe alphabet backwards and measure whether the user has accomplished thetask in a manner consistent with a certain level of sobriety orimpairment. Such tests may, in one implementation, be created asstandalone applications that can be used on demand without incorporationinto other aspects of the present invention.

The active sobriety tests may prompt the user as a text message does,and in some aspects may do so in a manner that is inconspicuous.Furthermore, in some aspects, each test may be constructed in a mannerthat takes no more than twenty to thirty seconds to complete. This fallswithin the timeframe that a text message may be sent or call may be madefurther decreasing conspicuousness. In another aspect, where a user hasa wearable PMD that projects an image viewable only to the user (such asGoogle Glass®), the test may be one that does not require gross movementat all, such as a test where a user follows a dot with his pupil orblinks the answer to a math problem. In some embodiments, the numberand/or rate at which the user makes errors in typing may be used as anindicator of the impairment of a person. Such tests may, in oneimplementation, be created as stand-alone applications that can be usedon demand without incorporation into other aspects of thepresent-invention.

In order for the PMD 510 to function optimally, in some aspects the PMD510 preferably gives any implementing program access to certainfeatures, services and data on the PMD 510. The PMD 510 may need access,for example, to the GPS and location services, accelerometer, gyroscope,ambient light sensor, biometric sensors, microphone, cameras, photographgallery and contacts. The PMD may also grant access to connectedsensors, such as those connected via Bluetooth. In some aspects, wherehealth or other sensitive information is accessed, such data may beencrypted, or an encryption system may be employed so as to protect suchdata and/or to comply with HIPAA and/or other government regulations.

The system 500 may need access to the location services of the mobiledevice 510 in order to detect when the user has entered a location suchas a bar, liquor store or other establishments where alcohol may beprocured. If the user remains in the location for more than apredetermined time interval (e.g., fifteen minutes), the system 500 maycheck the user into the location and assume that the user may bestaying. The user may be automatically checked out of the location oncethe user has left the location or geofence. The accelerometer, gyroscopeand ambient light sensor of the mobile device 510 are the sensors thatmay be leveraged to track measure and log the gait of the user. Thecameras of the mobile device 510 may be used to measure and track theuser's pupil dilation and movement speed, as well as the user's bloodpressure. Access to a photo gallery on the mobile device 510 may allowthe system 500 to strengthen the system's facial recognition and pupiltracking and measuring algorithms.

The feature requiring the most active input from the user is the BACcalculation feature. This feature allows the user to enter when theyhave had a drink and may track how many drinks the user has had overtime while simultaneously, calculating the user's BAC using, forexample, the user's gender and weight. The user may be able to input thedrink consumed with the device keyboard and as the user is entering thedrink, the system 500 may search its internal bartender guide databasefor a match. If a match in the database is found, the system 500 maypull the volume of alcohol from the drink recipe. If a match is notfound, which is unlikely, the system 500 may default to approximation,assuming that the drink was 1.25 oz. of 80 proof liquor, 12 oz. of beeror 5 oz. of table wine. Alternatively or in addition, the system 500 mayestimate the volume of alcohol based on a video of the drink preparationor by using chromatography to directly measure the volume of alcohol.Once the user has inputted the first drink the system 500 may continueto count the time intervals between drinks, setting the time of thefirst input as the top of the hour. From that point, the system 500 maybe calculating the time interval between the current and subsequentdrinks, and subtracting 0.01 BAC for every 40 minute interval betweendrinks.

With these different measures of intoxication signs recorded over time,from gait to pupil dilation, the instant system 500 may be able tocompare the data of one recorded sign of intoxication to each of thedata sets from the other sign categories in order to estimate theintoxication level of the person, or how far the person is from normal.This may be accomplished by comparing the determined levels ofabnormality to the normal levels detected at calibration and through thesystem's learning.

In some instances, when the system 500 is collecting data for thevarious signs of intoxication, the system 500 may be able to graph thedata of the perceived intoxication over time. As a result, the user oradministrator may be able to look at a graph generated by the system 500and learn approximately how intoxicated the user was at a given time.Each data point on the graph may retain additional information such asthe location that the user was at that intoxication measurement. As aresult, clicking a data point on the graph may present the user oradministrator with a breakdown of the available intoxication measuresfor that point, allowing the user or administrator to have a betterunderstanding of how the user progressed over a particular timeinterval. The system 500 may also present the data of the differentsigns, such as loss of fine motor control, loss of balance, slurredspeech or alteration in intonations, fluctuation in vital signs andchanges in pupil size and response times side by side, to createdashboard summary of how the user was progressing as the user consumedalcohol over time.

The system 500 also has the ability to send notifications topredetermined contacts. For instance, if the user is on parole, thesystem 500 would be able to notify the user's parole officer andappropriate department of the precinct. If the user is in arehabilitation program, the system 500 would be able to notify theuser's sponsor. Likewise, if the system 500 is being used voluntarily bya person trying to be more responsible, the system 500 can notify theuser's friends of the user's state of intoxication. For example thesystem 500 may send a user's friend a notification stating that “yourfriend is getting very intoxicated; you may want to check on them,”while simultaneously notifying the user with a message stating that “youare dangerously intoxicated.” Furthermore the system 500 may notify theuser when the user has reached the legal limit of intoxication.

In addition to the notifications, the system 500 can communicate withother devices. Just like mobile devices can communicate with cars overBluetooth to let the car know of an incoming message or call, amongother notifications, the instant system 500 can alert the vehicle,making it aware that the driver is unfit to drive because of their BAC.In some instances, the notification may even disable the vehicle,preventing the user of the system 500 from operating the vehicle.

In a non-alcohol related example, colon cancer risk or other digestiveissues may be better evaluated by measuring the time the person is onthe toilet as well as the amount and type of interaction with the deviceduring that period. Indeed, the time between when the user stopsinteracting with the device and when the device (via accelerometer orotherwise) determines the user is walking out of the bathroom may beused to determine the amount of cleanup required and therefore provide aproxy for the firmness of the user's stool. The sound of toilet paperbeing pulled and/or the sound of the toilet being flushed and/or thesound of the waste hitting the toilet water may all also be utilized todetermine certain health factors.

In one aspect, abrupt changes to measured data may indicate that thedevice has been stolen. In another aspect, multiple profiles may becreated for multiple users of the device. Such profiles may be sharedwithin affinity or other groups. For example, a husband and wife mayperiodically hold the phone of the other, and the phone would recognizethat it is now following the pattern of data associated with the spouse.

Perceivable or not by the affected person, there are a host of effectsthat a drug such as alcohol may have on an individual. Some of themeasurable signs of intoxication are loss of fine motor control, loss ofbalance, slurred speech or alteration in intonations, fluctuation invital signs, nystagmus, and changes in pupil size and response times.Some of these signs are passively measurable, such as the loss ofbalance, which affects one's gait. Additionally, speech can be passivelymeasured. One factor relating to measurability is whether the datapresents itself in similar repetitive cycles resulting in a sinewave-like function, although the absence of such cycles does not make asign of intoxication unmeasurable.

Exemplary Systems/Devices for Determining the Level of Impairment

Aspects of the present invention also relate to systems and devices toestimate the level of impairment of a person. In an exemplaryembodiment, the device comprises (a) one or more sensors operablycoupled to a first PMD, the sensors configured to capture ambient dataabout a person, wherein the first PMD is configured to query at leastone data base to identify baseline data and compare the ambient data tothe baseline data to determine the likelihood that the person isimpaired.

Referring now to FIG. 6, therein is shown an embodiment of a system 600for estimating the level of impairment of a person. In the embodiment ofFIG. 6, a smartphone (first PMD) 605 having internal sensors 608 isoperably coupled to external sensors 606 and 607, external measuringdevice (breathalyzer) 621, camera 623, computing device 625 and remoteserver 631. The internal sensors 608 of first PMD 605 may comprise oneor more sensors, including, but not limited to a fingerprint sensor, anaccelerometer, a three-axis gyroscope, a compass, an assisted GPS, acharging sensor, a battery measurement sensor, an ambient light sensor,a magnetometer, an proximity sensor, an orientation sensor, and/or oneor more biometric sensors.

In some embodiments, other data collection devices may also be integralto the first PMD 605. For example, the first PMD 605 may also comprise afront facing camera (with associated sensors), a rear facing camera(with associated sensors), a microphone, a digitizer (e.g., a touchscreen), etc. Further, the first PMD 605 may have capabilities tocapture other data, including, but not limited to cellular voice,cellular data (e.g., usage history), WiFi data (e.g., internet sitesvisited) and/or Bluetooth data, and store and/or transmit data.

The first PMD 605 may optionally be equipped with one or more of avariety of other sensors, including, but not limited to a wirelesscharging station (and associated sensors), terrestrial/RDS/satelliteradio sensors, pressure sensors, temperature sensors, humidity sensors,CO sensors, CO₂ sensors, NFC communications sensors, face and objectdetection devices (e.g., within software supporting camera), and/orbarometric pressure sensors.

In the embodiment of FIG. 6, each of first and second external sensors606, 607 may be one of the many types of sensors listed above as sensorspossibly internal to the first PMD 605, except that external sensors 606and 607 reside external to the PMD. Additionally, optional externalmeasurement device (breathalyzer) 621 is also coupled to first PMD 605.Although the external measuring device 621 of FIG. 6 is a breathalyzer,in other embodiments, one or more such external devices may include, butare not limited to devices to measure blood alcohol level, blood sugarlevel, blood pressure, heart rate, activity level, calorie consumption,etc.

In the embodiment of FIG. 6, the first PMD 605 is operably coupled to acomputing device 625, in which data may be analyzed, identified and/orstored. However, in other embodiments, the data may be analyzed,identified and/or stored on first PMD 605 and/or the first PMD 605 maybe operably coupled to one or more servers 631 or other storage devices(e.g., a zip drive, thumb drive, CD ROM, DVD, etc.) from which the firstPMD 605 may pull useful data. Additionally, the first PMD 605 may beoperably coupled to one or more cloud storage databases (not shown).Operably coupling may comprise, directly coupling the external sensors,measuring device(s), servers and/or storage devices or may comprisecoupling via near field communication, Bluetooth, local area network,Wi-Fi network, cellular network, wide area network, etc.

Although not shown in the embodiment of FIG. 6, the system 600 may alsocomprise a second PMD operably coupled to the first PMD 605, wherein thesecond PMD and the first PMD 605 share data with each other. In someembodiments, the second PMD is configured to process the ambient datashared by the first PMD 605. In some embodiments, a plurality of PMD'smay be operably coupled to the first PMD 605, and the plurality of PMD'smay share data with each other and/or the first PMD 605. In someembodiments, one or more of the plurality of PMD's may be configured toprocess the ambient data shared by the first PMD 605.

In some embodiments, the first PMD 605 may also comprise a transmitter,and the transmitter may be configured to transmit notifications to oneor more predetermined contacts. For example the system 600 may send auser's friend a notification stating that “your friend is getting veryintoxicated; you may want to check on them,” while simultaneouslynotifying the user with a message stating that “you are dangerouslyintoxicated.” Furthermore the system 600 may notify the user when theuser has reached the legal limit of intoxication.

The foregoing descriptions of specific embodiments of the presentinvention have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations are possible in light of the above teaching. Theembodiments were chosen and described in order to best explain theprincipals of the invention and its practical application, to therebyenable others skilled in the art to best utilize the invention and thevarious embodiments and modifications as are suited to the particularuse contemplated. It is intended that the scope of the invention bedefined by the components and elements described herein and theirequivalents.

What is claimed is:
 1. A device for detecting changes to a userinterface, the device comprising: one or more sensors operably coupledto a primary monitoring device (“PMD”); the sensors comprising one ormore of a touch screen, an input device, a gyroscopic sensor, anattitude sensor, a microphone, a camera and a motion sensor; wherein thePMD monitors at least one of the sensors to detect user errors ininteraction with the device; comparing an error rate with database datacontaining one or more of measurements of error rates of other persons;past error rates of the current user; projected error rates based onphysiological characteristics of the user, and/or any of the foregoingwhen read in an environment with comparable amount of ambient noise; andidentifying a level of difference between the database data and theerror rate.
 2. The device of claim 1, wherein the error rate is comparedto a second database, which database contains physiological changes andassociated error rates (the “Association”).
 3. The device of claim 2,wherein the user is alerted to one or more of the physiological changesidentified by the Association.
 4. The device of claim 2, wherein one orboth of the error rate and the Association is provided to at least oneof the user, a drug recovery program sponsor, and a health careprovider.
 5. The device of claim 2, wherein one or both of the errorrate and the Association is provided to a parent or a guardian of theuser.
 6. The device of claim 2, wherein the user is alerted to one ormore of the physiological changes identified by the Association when thephysiological change is greater than one sigma deviation from normalwhen compared to one or more database measurement sources.
 7. The deviceof claim 6, wherein one or both of the error rate and the Association isprovided to at least one of the user, a drug recovery program sponsor,and a health care provider.
 8. The device of claim 1, wherein errormeasurement is compared to data limited by one or more of location,ambient sound, or time of day.
 9. A device for detecting changes toverbal cadence, the device comprising: one or more sensors operablycoupled to a primary monitoring device (“PMD”); the sensors comprisingone or more of a microphone, a gyroscopic or non-gyroscopic devicecapable of measuring movement caused by sound waves; wherein the PMDmonitors at least one of the sensors to detect the rhythmic and/orwave-like function (the “Cadence”) generated by a user's voice;comparing the Cadence with database data containing one or more ofmeasurements of other persons; past measurements of the current user;projected measurements based on physiological characteristics of theuser; or measurements of other users; and identifying a level ofdifference between the database data and the Cadence (the “Change”). 10.The device of claim 9, wherein the Change is compared to a seconddatabase, which database contains physiological changes and associatedmeasurements of a change to voice cadence (the “Second Association”).11. The device of claim 9, wherein the Change is determined only bycomparison to database entries made in an environment with a comparableamount of ambient noise.
 12. The device of claim 9, wherein the user isalerted to one of more of the physiological changes identified by theSecond Association.
 13. The device of claim 9, wherein one or both ofthe Second Association and the Change is provided to one or both of theuser and a health care provider.
 14. The device of claim 9, wherein oneor both of the Second Association and the Change is provided to a parentor guardian of the user.
 15. The device of claim 9, wherein the user'svoice detected is compared to data regarding voice patterns to determinean identity of the speaker.
 16. A device for detecting changes to auser's balance, comprising: one or more motion sensors operably coupledto a PMD; wherein the PMD is configured to monitor and save data fromthe sensors, where such data is consistent with a fall or other loss ofbalance (an “Episode”); determining whether a number of Episodes takingplace over a set period of time exceed a threshold number; where thethreshold number is different for different levels of magnitude ofEpisodes.
 17. The device of claim 16, wherein the different levels ofmagnitude of Episodes requires at least a one sigma deviation fromnormal.
 18. The device of claim 16, wherein the different levels ofmagnitude of Episodes requires at least a two sigma deviation fromnormal.